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REMARKS 

Entry of the amendments to the specification, claims and 
abstract before examination of the application is respectfully 
requested. These claims patentably define over the art of record. 

If there are any questions regarding this Preliminary 
Amendment or this application in general, a telephone call to the 
undersigned would be appreciated since this should expedite the 
prosecution of the application for all concerned. 

It is respectfully requested that, if necessary to effect 
a timely response, this paper be considered as a Petition for an 
Extension of Time sufficient to effect a timely response and 
shortages in other fees, be charged, or any overpayment in fees 
be credited, to the Account of Crowell & Moring LLP, Deposit 
Account No. 05-1323 (Docket #420LFK/50041 ) . 



CROWELL & MORING, LLP 
P.O. Box 14300 
Washington, DC 20044-4300 
Telephone No.: (202) 628-8800 
Facsimile No.: (202) 628-8844 
GRE : kms 



Respectfully submitted. 
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METHOD AND APPARATUS FOR LOCATING OR REDISCOVERING PARTIALLY 

OBSCURED OBJECTS IN IMAGES 

BACKGROUND AND SUMMARY OF THE INVENTION 

This application claims the priority of German patent 
document 199 02 681.5, filed 23 January 1999 (23,01,99) and PCT 
International Application No. 99/DE99/04Q64 , filed 22 December 
1999, the disclosure of which is expressly incorporated by 
reference herein. 

This invention relates to a [process for finding] method and 
apparatus for locating or rediscovering partially obscured 
objects in images^ [again] by correlation of an object reference 
with image values_2_ as well as a system for rediscovering 

objects in images.] 

Numerous practical uses require objects to be automatically 
rediscovered in images or digital image sequences. But it can 
happen that parts of the object are covered up or obscured by 
obstructions in the field of view . 

In a ]cnown process for that facilitates the] rediscovery 
of objects in images, the object image to be [searched for] 
located stores a reference and that reference is subsequently 
correlated with the image within a limited range. Only part of 
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the reference is used during correlation and the image [dots] 
pixels of that reference belong to the object. Parts that do not 
belong to the object are masked [up] by object windowing or 
object masking. The [place] location of the correlation maximum 
supplies the current position of the object in the image^, because 
the image [dots] pixels there agree with the object pixels. 

This known process however [entails the following problem:] 
is problematic in that the correlation no longer provides the 
correct position of the object when ijt, [the object] happens to 
be partly covered up , because [. Here is why:] one correlates in 
the covered area with image [dots] pixels that do not belong to 
the object. Due to this coverup, only part of the object is 
compared to the reference, [something that] which leads to an 
inaccurate recognition of the object or of the position of the 
object. Imprecise recognition of the coverage area very quickly 
yields an erroneous object position in the image. 

It is therefore [the] a purpose of [this] the present 
invention to provide a process for rediscovering objects in 
images, which will facilitate precise recognition and precise 
determination of the object position. 

Another object of the invention is to provide [Furthermore, ] 
a system [is to be provided] for the rediscovery of objects in 
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images, with which the object position can be determined 
precisely . 

These and other objects and advantages are achieved by the 
method and apparatus [This problem is solved by the process] for 
[the] rediscovery of objects according to the invention, which 
includes [in images according to Claim 1 and the system for 
rediscovering objects in images according to Claim 6. Other 
advantageous features, aspects, and details of the invention will 
emerge from the subclaims, the specification, and the drawings. 

The invention-based process for the rediscovery of objects 
in images comprises] a correlation of an object reference with 
image values . If [where - if] an object is partially obscured 
Xcovered up) , [-] the image values [,] that are within the 
coverage area 'in the image, [will be] are replaced by gray values 
of the object or of [the] an object reference, before the 
correlation is performed. In that way, [one can considerably 
reduce] the errors connected with the rediscovery of the object 
are significantly reduced . 

As part of this process, a reference image is preferably 
subjected to interference windowing in order to replace the image 
values within an interference mask by the gray values of the 
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object. Advantageously, the reference image is subjected to 
object windowing in order to get the object reference. 

In the present invention^, [at hand, ] especially prior to 
coverup, [the] an image of the complete object is stored^ [in 
order] and is used to determine, the position of the object in 
case of a partial • coverup . Advantageously, covered parts of the 
object in the image are replaced by parts of a stored reference. 

The invention-based system for the rediscovery of objects 
in images [comprises] includes a camera to taJce a picture, an 
image data memory to store an object reference, an image data 
processor to replace [the] gray values that lie in the image 
within a coverup area[,] with gray values of the object 
reference, and a correlation unit that correlates the image - 
altered by the image data processor - with the object reference. 
The invention-based system is able precisely to recognize the 
coverup area and to determine a precise object position in the 
image. Errors are avoided or [extensively] substantially reduced 
in that the already stored information on the appearance of the 
object is employed. 

The invention is guided by a basic cybernetic idea: the 
system stores the image of the complete ob j ect and uses this 
information to [find out] determine what is behind the coverup. 
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[The basic cybernetic idea] This technique is patterned after 
[guided by] the human observer^^ who remembers the image of the 
complete object and who has an idea of what is behind the 
coverup- [This basic cybernetic idea is used in this invention 
and is expressed in technical terms, as] As a resultjt. [of which] 
one can [very precisely] determine precisely the position of the 
object in case of partially covered objects. 

Other objects, advantages and novel features of the present 
invention will become apparent from the following detailed 
description of the invention when considered in conjunction with 
the accompanying drawings . 

BRIEF DESCRIPTION OF THE DRAWINGS 

[The invention will be described below by way of • example and 
with reference to the drawings.] 

Figure 1 graphically illustrates the individual steps of the 
invention-based process; and 

Figure 2 is a flow chart of the invention-based process. 
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DETAILED DESCRIPTION OF THE DRAWINGS 

Figure 1 shows the steps of the invention-based process by 
way of example, on the basis of target tracking. [First of all, 
one can see a] The target or object 1 is initially visible, but 

[that, ] as time goes on, it disappears behind an obstruction [a 
coverup] 2 , which covers it up . At a certain point in time, one 
or [several] more parts la of object 1 can still be recognized, 
while the remaining parts of object 1 are behind [coverup] the 
obstruction 2. If a picture is taken at [At] that point in time, 

[a picture is taken and] it contains object 1 which is partially 
covered by [coverup] obstruction 2. 

From a known reference image, [in] which [is contained] 
contains the target or object 1, [one now produces] an object 
mask 10 is produced ; Object mask 10 is [supposed] superimposed 
with the picture that was taken and that contains target 1 behind 
[coverup] obstruction 2. [That] This can be done so long as part 
la of object 1 can still be seen in the image. By [superposition 
with] superpositioning object mask 10 with the partially obscured 
object and the obstruction , [one gets] an interference mask 20 
is created/ which is superposed on [coverup] obstruction 2. That 
iS/ interference [Interference] mask 20 consists of [contains] 
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those parts of [coverup] the obstruction 2 that cover the 
remaining parts of the object 1 in the image. 

By means of an image data processor, the image [dots] pixels 
inside interference mask 20 are replaced by the gray values of 
[odds] obi ect 1 that are taken from the reference image. This 
results in [One gets] a replacement 21 with image [dots] pixels 
that correspond to [the] just the covered parts of object 1. 
Replacement 21 is then inserted in the picture that was taken in 
the area of interference mask 20 so that the image now generated 
contains the complete object 1, without any parts of object 1 
missing. The image, thus generated with the help of the image 
data processor, forms a foundation for further procedural steps. 

From the stored reference image [, ] J_that, as described 
above, contains object 1J_, [one gets] a reference or object 
reference 11 is created, with the help of object mask 10. Object 
reference 11 is now correlated with image 22 that contains 
replacement 21. The correlation maximum is determined by means 
of a peak detection- The location [place] of the correlation 
maximum thus supplies the precise position of object 1 in the 
picture , 

Figure 2 is a [flow chart showing] block diagram which shows 
the process [steps] for the automatic [finding] location or 
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rediscovery of object 1 in the picture that was taken. The data 

of a reference image 201 are subjected to interference windowing 

202. and [- This is followed by the replacement of] those image 

parts 203 [-] in which the object is behind the [coverup -] 

obstruction are replaced with the gray [value] values of the 

[objects] object from the reference image in block 204 . 

[Furthermore, the] The reference image is also subjected to 
object windowing 205 that, as a result, supplies the object 
reference. In a correlation [step,] unit 206, the object 
reference is correlated with the image that contains the 
replacement from block 204 . [Last, one performs the peak 
detection] FinallV/ in order to determine the correlation maximum 
and the position of the object , a peak detection 207 is 
performed . J.In the process, the reference image for example is 
a previously taken picture that contains object 1.1. 

In the method arid apparatus according to the invention^ [at 
hand, ] in the search area of the image, the gray values of the 
[coverup] obstruction are replaced by the gray values of the 
object; as a result, [one gets] considerably increased precision 
of recognition and position determination of objects is achieved . 
Moreover, there [There] is considerably less intervention in the 
signals that are subject to correlation than during the out- 
masking of entire image portions. Errors during position 
determination are reduced considerably [reduced] by storing the 
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previously obtained information on the appearance of the object 
and by using this information to find the object behind [a 
coverup] an obstruction . 

The foregoing disclosure has been set forth merely to 
illustrate the invention and is not intended to be limiting. 
Since modifications of the disclosed embodiments incorporating 
the spirit and substance of the invention may occur to persons 
skilled in the art, the invention should be construed to include 
everything within the scope of the appended claims and 
eguivalents thereof . 
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•Process and System for Rediscovering Objects in Images Again 

This invention relates to a process for finding objects 
in images again by correlation of an object reference with 
image values, as well as a system for rediscovering objects in 
images . 

Numerous practical uses require objects to be 
automatically rediscovered in images or digital image 
sequences. But it can happen that parts of the object are 
covered up. 

In a known process, that facilitates the rediscovery of 
objects in images, the object image to be searched for stores 
a reference and that reference is subsequently correlated with 
the image within a limited range. Only part of the reference 
is used during correlation' and the image dots of that 
reference belong to the object. Parts that do not belong to 
the object are masked up by object windowing or object 
masking. The place of the correlation maximum supplies the 
current position of the object in the image because the image 
dots there agree with the object pixels. 

This known process however entails the following problem; 
the correlation no longer provides the correct position of the 
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object when the object happens to be partly covered up. Here 

is why: one correlates in the covered area with image dots 

that do not belong to the object. Due to this coverup, only 

part of the object is compared to the reference, something 

that leads to an inaccurate recognition of the object or of 

the position of the object. Imprecise recognition of the 

coverage area very quickly yields an erroneous object position 

in the image. 

It is therefore the purpose, of this invention to provide 
a process for rediscovering objects in images, which will 
facilitate precise recognition and precise determination of 
the object position. Furthermore, a system is to be provided 
for the rediscovery of objects in images, with which the 
object position can be determined precisely. 

This problem is solved by the process for the rediscovery 
of objects in images according to Claim 1 and the system for 
rediscovering objects in images according to Claim 6. Other 
advantageous features, aspects, and details of the invention 
will emerge from the subclaims, the specification, and the 
drawings . 

The invention-based process for the rediscovery of 
objects in images comprises a correlation of an object 
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reference with image values where - if an object is partially 
covered up - the image values, that are within the coverage 
area in the image, will be replaced by gray values of the 
object or of the object reference, before the correlation is 
performed. In that way, one can considerably reduce the 
errors connected with the rediscovery of the object. 

As part of this process, a reference image is preferably 
subjected to interference windowing in order to replace the • 
image values within an interference mask by the gray values of 
the object. 

Advantageously, the reference image is subjected to 
object windowing in order to get the object reference. 

In the invention at hand, especially prior to coverup, 
the image of the complete object is stored in order to 
determine the position of the object in case of a partial 
coverup. Advantageously, covered parts of the object in the 
image are replaced by parts of a stored reference. 

The invention-based system for the rediscovery of objects 
in images comprises a camera to take a picture, an image data 
memory to store an object reference, an image data processor 
to replace the gray values that lie in the image within a 
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coverup area, with gray values of the object reference, and a 
correlation unit that correlates the image - altered by the 
image data processor - with the object reference. The 
invention-based system is able precisely to recognize the 
coverup area and to determine a precise object position in the 
image. Errors are avoided or extensively reduced in that the 
already stored information on the appearance of the object is 
employed. 

The invention is guided by a basic cybernetic idea: the 
system stores the image of the complete object and uses this 
information to find out what is behind the coverup. The basic 
cybernetic idea is guided by the human observer who remembers 
the image of the complete object and who has an idea of what 
is behind the coverup. This basic cybernetic idea is used in 
this invention and is expressed in technical terms, as a 
result of which one can very precisely determine the position 
of the object in case of partially covered objects. 

The invention will be described below by way of example 
and with reference to the drawings. 

Figure 1 graphically illustrates the individual steps of 
the invention-based process; Figure 2 is a flow chart of the 
invention-based process. 



4 



Attorney Docket No. 420LFK/50041 
Translation of PCT International 
Application No. PCT/DE9 9/04064 

Figure 1 shows the steps of the invention-based process 
by way of example, on the basis of target tracking. First of 
all, one can see a target or object 1 that, as time goes on, 
disappears behind a coverup 2. At a certain point in time, 
one or several parts la of object 1 can still be recognized, 
while the remaining parts of object 1 are behind coverup 2. 
At that point in time, a picture is taken and it contains 
object 1 which is partially covered by coverup 2, 

From a known reference image, in which is contained the 
target or object 1, one now produces an object mask 10. 
Object mask 10 is supposed with the picture that was taken and 
that contains target 1 behind coverup 2 • That can be done so 
long as part la of object 1 can still be seen in the image. 
By superposition with object mass 10, one gets an interference 
mask 20 which is superposed on coverup 2. Interference mask 
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20 contains those parts of coverup 2 that cover the remaining 
parts of the object 1 in the image. 

By means of an image data processor, the image dots 
inside interference mask 20 are replaced by the gray values of 
odds 1 that are taken from the reference image. One gets a 
replacement 21 with image dots that correspond to the just 
covered parts of object 1. Replacement 21 is inserted in the 
picture that was taken in the area of interference mask 20 so 
that the image now generated contains the complete object 1, 
without any parts of object 1 missing. The image, thus 
generated with the help of the image data processor, forms a 
foundation for further procedural steps. 

From the stored reference image, that, as described 
above, contains object 1, one gets a reference or object 
reference 11 with the help of object mask 10. Object 
reference 11 is now correlated with image 22 that contains 
replacement 21. The correlation maximum is determined by 
means of a peak detection. The place of the correlation 
maximum thus supplies the precise position of object 1 in the 
picture . 

Figure 2 is a flow chart showing the process steps for 
the automatic finding or rediscovery of object 1 in the 
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picture that was taken. The data of a reference image are 

subjected to interference windowing. This is followed by the 

replacement of those image parts - in which the object is 

behind the coverup - with the gray value of the objects from 

the reference image. Furthermore, the reference image is 

subjected to object windowing that, as a result, supplies the 

object reference. In a correlation step, the object reference 

is correlated with the image that contains the replacement. 

Last, one performs the peak detection in order to determine 

the correlation maximum and the position of the object. In 

the process, the reference image for example is a previously 

taken picture that contains object 1. 

In the invention at hand, in the search area of the 
image, the gray values of the coverup are replaced by the gray 
values of the object; as a result, one gets considerably 
increased precision of recognition and position determination 
of objects. There is considerably less intervention in the 
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signals that are subject to correlation than during the out- 
masking of entire image portions. Errors^ during position 
determination are considerably reduced by storing the 
previously obtained information on the appearance of the 
object and by using this information to find the object behind 
a coverup. 
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